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^ There are two* clusters of research reports in this issue of ISE. 
The first cluster, PIAGETIAN STUDIES, contains seven studies. The pre- 
dominance of Piagetian research analyzed in this issue appears to reflect 
a current ti;end in science education. The second cluster, INSERVICE 
EDUCATION, contains two studies which examine the outcomes of inservice 
education programs. 

This issue als*o contains, for the first time, a response to an 
analysis of a research report. W^believe this kind of exchange is 
beneficial to the profession and strongly encburage others to send us 
their publishable responses to critical analyses. Such responses need 
not be from the author of the research »tudy being analyzed. 
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PIAGETIAN STUDIE~S 
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I 



Kolodiy, GGor[>e. "The Coj>,nil;ive Dovelopnent of IIif>,h School and Collep;e 
Scienc*e Students." Tl\c rionrna l of Co] lp_< >c 'Science Teachi ng:, 5(1): 
20-22^^ September, 19 75. v * ' "> , * . 

. • De^riutors — "^Cof^rt i tive ' DevelDOTricnt i*. '"Colle<?,e Students; Hicther 
Education; -Intellectual Development: Pve<iiction: Science 
Education; Secondary Education;' *Secondi^ry School Science 

Expanded Abstract and Analysis Prepared Especially for I.S.E. by Gene F, 
Craven, Oregon State University. 



^ Purpose ' ? ' 

Tlie'stated purpose o^ this study w^s "to obtain indications of changes in. 
mental level from high school thronjgh college and to investigate .the relar 
tionship of cof^nitive level to SAT scores^^ college grades, and attrition . " 
(Kolodiy, 1975; p.'^20) 



* ' ■ . V 

Ratio nale , ' , ^ • 

— — ♦ * , ' 

Piagetian thepry . continues to make important contributions to our Jcnowledp;e 

of cognitive development. In 'the past decade s tudies of the child's acquis' 

sition of knowledge- have broadened into formaf education. It is 6nl»y 

recently, however, that reseai|chers have lo'bked at the relevance of Piage- ^ 

tian theory for high school and college student learning. \ ^^'^ 

/ , . . ■ • . ( • 

'I 

Piaget Views human cognitioit as a 'comDlex process of int,eractin?> with t:!^^ 
outside world that includes two siiijultaneous and complementary aspects 
which he refers to as assimilation and accoinmodati on . By* assimilation he « 
means actively selecting and interpreting external objects and events in 
terms of the learner's mental strueture (his oresently ^vailabl>^nd favored 
way -Qf^thinking about thin^^s) » Accommodation means adapting one's^own men- 
t^l structure by "taking cognitive' account of the various real properties 
and relationshfps- among properties that external objects and eventjTlTDssess ." 
(Flavell, 1977; p. 7. Jhus Paapet's assimilation-accommodation mAdel pro- 
vides a useful ve!iLcle'for VhinkiYi?^ about liow a persop^s cognitive bvsiem ^ 
•might gradually evolve with rtiaturatioa and experience/. 
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Four sequential starves of cor.nitivo development are postulated in Pia?>e-w 
tian tliebry; natrelv', sensorimotor, pre-operational , concrete-operational, 



and formal 



-opeijat 



jational sta^.es, ' ifach sta^e is characterized by an abrlity 



'to perform certain mental functions and an inability to perform others 
(Kolodiy, 1975 ; Inhelder, 1974; fiaget, 1972). • . - • ' 

Studies of the cognitive development of lijtL'gli scliool and college students, 
including the study analyzed for the present abstract, usually pertain to 
the concrete- and formal-operational stap,es which typically begin between 
ages 7-11 and 11-15 years, respectively.' ^ 

• 1 . • ^ . ' . 

\^en oSjects or events are used to provide a* problem' situatiqn , the learner 
at the concrete-operational stag(? of cognitive development characteristi- 
cally focuses on the perceptible^and inferable reality in front of him, 
Various skills are used to orde^Xand interrelate whatever proper ties^ or 
^features of the situation he or she ^an detect. His or her "conceptual 
approach generates |olution attemnts that are far more rational and tasK- 
relevant than tfiose produced by the pre-onerational learner. It does, 
however, pertain rather closely to detected empirical reality, and specu- 
lations about other possibilities occur only with difficulty and as a last 
resor^" (Flavell, 1977/p* 103)* . 

A learner at the formal-operational s.tap,e*^!rs more apt to begin with possiA 
bility and only subsequently proceed to reality. He or she may examine the 
problem situation carefully and try to determine what all oii the possible 
solutions or states of affairs might be, and then systematically try to 
discover wfiich of these is,, in fact, the real one for the present case. 
F*or the formal-operational thinker, realitv is seen as that particular por- 
tion of a much wider world of nossibility which happens to exist or hold 
tfijie in a ^iven problem situation (Flavell, 1977). . - 
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The formal-operational thinker in.qnects the orohlem data, h^rpothe- 
sizes that such and such a theory or oxplancHrion micjit he the 
cc^rrect ohe , deduces from it that so and so empirical phenomena 
ought logical] y to occur or not occur, and then>tosts his theory 
by seeing if ;:hese predicted nhonomena do in fact occur. 



/ 



If .you think you have iust 4iGard^.<i dcsoription of t;extbook 
j&cien'tif ic roa^Joninf^, vou arc abs6lutelv rip.ht. *Bccauf^c of 
it^ heavy trade in hypotheses a4ii^lor^^ ^<'il dedudtions from * » 
hypetheses, it is called hvpothetico-dcductive reasoning, and 
it contrasts sliarply \;it*n the much more non theoretical arid 
nonspecula tive empiricfrinduc ti ve reasoning of concrete- 
operational thinkers (Flavell, 1977; pp. 103-104). 



Research Design and Procedure 



Twb tasks, the solution of which required formal thinking, were presented 
to a total of 70 subjects who represented three distinct populations. One 
group consisted of 20 high school sophomores selected ^rom an elective 
second-year BSCS biolog'^ course that was designed for students planning 
to pursue careers ;in science., A second group consisted of 25 college 
men selected from introductory courses in pliysics land natheiTiatics desi'gned 
for'science and engineering majors. Th£5 thir^d group consisted of 25 college 
seniors, maioVing in the sciences. 



SAT verbal and math scores were used as a basis for determining equivalency 
of the three groups. Since SAT scores werc^not available for the high 
school sophomores, scores of the previous year's graduating class were used 
to represent this group. 



\ 



The test instrument consisted of two Piagetian tasks. One task titled 
'*Combinat ions of Colorecl and Colorless Chemical bodies" has been used by 
Piaget and other inve's ti gators (Inhelder, 1953) to measure ability to use 
combinatorial logic. Tlie ojKperimenter , in the presence of the subject, 
adds a few drops of a solution to a test tube containing a colorless mix- 
ture produced prip-r to the beginning .of the experiment and notes the 
formation of a yellow color, llie subject is then presented with-five^ 
beakers of colorless liquids labeled g^ 1, 2, 3, and k. He or she is asked 
to reproduce the color and to find the ^function of each liquid. 
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A second task was entijLled ^'Hauling Weight on an Inclined Plane'^ (Inhelder, 
1958). Tlie apparatus for the tar,k consif^tod of a variable-height plane, a 
' roller skate to which weights could be added or subtracted, a hanging 
^« Voight, and a set of weights. The skate was set in equil-ihrium at the 



same an^;le on the plana. The subibcl was than asked to identify tlie varia- 
bles that make up. the equilibrium, aiui to deternine the relationship Kinon g 
them. I , * 

The percentages of students at each level (unpet concrete, lower formal, 
♦upper formal) were summarized in tabular form. Differences between the 

* three groups were analyzed via the noirpajjametric chi-square test with 
two degrees of freedom. This test provided the percenta^.e of students at 
'the formal level aria below formal level. Tlie expected frequencies were 
calculated from the criterion that the percentage o^ students scoring at 
the forma^^evel in each group should eq'ual the total percentage of all 

...^i^^hree^ groups scgrin'g at tfiis leve^.' Also, a correlational .analysis of all 
the variabJLes was carried out. 



Findings ^ , ^ ^ ' 

1. The percentage ofjstudents at the upper formal level was roughly the 
same (35 and 32 percent) for high school sophomores and college fresh- 
men, while somewhat higher (64 percent) for ^he college seniors. 

f • - ♦ 

2. * .Using the analoc^ of Scheffe's theorem for chi-square, it was found 

that "the high scliool and college freshmen samples were equal and 
jointlj/^if fereat from the^ college senior sample" at the (f.05 level 
o^^ignif icance. 

3. A correlational anal5?'sis of variance revealed that Scores on'the 
Piagetian* tasks were significantly correlated to SAT math scores but ' 
not to college grades which correlated significantly to SAT verbal 
scores, " 



A follow-up the 25 college fi;eshmcn subjects^showed that seven had 
failed to complete the second semester or nhysics. Each of the seven 
had scored as non formal on the Piage^t tasks. ' j 



§4 ' . 



X 



Inter]tr o tat i oil s * ' ^ ^ , 

The investiTvitor conclucfed that a) a mnjority *of hi school and college' 

students are below the formal level in cop^nitive functioning; b) collep,e 

science educati€)h d^cs not raise cognitive level but «rather eliminates' 

'nonformal students through attriti(5n; and,c) college grades are more 

< 

closely correlated to verbal ability than to cognitive functioning. He 
reasons that teachers should be awafre of their Students' level of cogni- 
tive functioning, and plan educational experiences accordingly^. Also, 
that lecture me^thods must be balanced by more concre^te activities where 
students caa engage in manipulation of materials and verbal explanation^, 
among themselves. • < ' 



ABSTRACTOR'S AI^ALYSIS ^ * 

f * 

/ 

Since the primary purpose of this investigation was to' ^'obtain indications 
of changes in mental level t]pom high school through college," a longitudi- 
nal study that would have monitored the cognitive development? of a given 
group of subjects over this four to eight-year, period v;ould have been 
optimal. Unfortunately, the^time restrictions placed on most^inves tigators , 
parti(^ularly^ doctoral^ca^didates , mak^ longitudinal studies impractical,. 
Tlius, a status study of equivalent grouos of subjects is a reasonably valid 
practical alternative to a longitudinal study. 

■ • t 

Tlie present study is best classified as a ''Static-Group CompariS(|n" design 
according 'to Campbell and Stanley's criteria for designs for research (196^3, 
p. 182). This pre-experimen^tal design requires equivalent groups that are 
tested after having experienced different .treatments • A major problem with 
■ the. "Static Group- Comparison" studies is that"of determining that the groups 
are, in fact, etjuivalent. * 

^SAT scores were used, to establish group equivalency in the present study, 
a procedure that raises several questions relative to the ^ ctual equiva- 
lency ^of the groups-. First, is this single critorian an adequate basis. for 
determining ^^quivalcncy when SAT scores -v^ere , in fact,, not available for 
the high school sophomores? How valid is the use of SAT test scores of the 
previous year ' s. graduating class (oj" the entire scitool?) to represent the 
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scores 'of a >',roup of 20 sophomores in an elective class for students 
planning collepc-pi'epara toty careers in yience? Are equivalent nean 
SAT scores indeed equivalent when the tests wore triken ovei? a period 
of four to six vcarsl during: which itational mean SAT scores declined 
annually? ' ( ChroniCc^lW )f Hif>hor e ducation, Sept. *15 , 1975; 18).- 

r • 

Second, were the subjects of the three groups from the sane p^eographical 
region and from comparable socio-economic backgrounds? Had tbev attended 
comparable elenentary and secondary schools? Riaf^et ret)orts that the speed 
of cogni-tive development can vary from one environment tfo toother, a fact- 
that is corroborated by other -researchers (Piage't, 1972). 

Apparently Kolodiy assumed that the ages of Ihe. subjdcts is r^ot a si^jnlfi* 
cant variable^ sybsequent to the sophomore year (15 or 16 years). This may 
be questioned since Placet has come to realize that the results lie found 
with academically able adolescents from the better schools in Geneva who, 
at 15 and 16 years were '"demonstrating formal reasoning, cannot be general- 
i^ed to other adolescent populations. He coit'cludes that when it ^comes to 
formal thought, there may be a retardation, in its formation to beoS^n 15 

4 * 

and 20 years when frequency ^nd quality of intelle<:;tual stimulation 
received from adults is inadequate (Pia.f^,et, 1972 ; pp, 6-9), 



An important omission of the journal' article is a description of the basis 
for selectA^JBfie schools that the subject^were attending and the method 
of selecting the subjects *from the universe of students in the classes . 
from which the subjects were selected. Inferential statistics aiffe based 
on t]\e assumpti(fh of random sei^^c^tion of sub'fects from a population to 
which the findings are to be generalized. For the findings of the present 
study tQ be generalized validly to "the .majority o^ high school and college 
students*' (Kolodiy, 1975: p. 22), ijandom selection of the subjects from 
this pop^ulati^on would be essential. 

The two tasks used as "Tlje Test Instrument" have been used by Piaget and 

ability to use combinatorial logic, 
and b) measure^ ability to itlentify^he variables in an experiment: t^x^o 
aspects of fornul reasoning. UHnle no minimum number of tasks has appar- 
ently been' cs4.ablished for a valid and reliabl(?^Piage tian test instruinent, 
two tasks would appear to he a minimal number from which to infer* three 
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levels of cognitive devolopment .* Chiapetta's (1955) recent review* of 
Piagetian studies relevant', to science instruction at the secondary and 
college level reveals that three 5r more tasks are commonly used. 

Although a general description of. the tasks is presented in the article, 
the reader is not informed as to who condudted the interviews, how this 
person (these persons?) was trained,- or details of the interview t^^fife*^^"""^ 
nique. Was a pilot study conducted fco perfect the interview -^fretrlmique , 
to learn of the type? of responses to exoect, and to te^st the data- 
recordinf> techniques?. Were audio- or video-tapes used to rei^ord the inter- 
views for subsequent analysis? In what order were, tha three g?roups of sub- 
jects tested and ove^ what period of time? Wliat criteria were used to 
classify the subjects /as upper concrete, lower formal, or upper formal? 
While these questions may appear trivial ^ their answers wbuld be essential 
to investigators desiring' to repli'cate the present study. 

The nonparame trie chi-square test is appropriate for determining whether or 
not a difference existed among the groups and the correlation analysis if 
the variables studied provides valid information so long as^ no cause-and- 
effect .relationship is inferred. 

Kolodiy's conclusions are qualified, in the text of the article by the phrase 

"within .the Limits of* this study." Yet, he tends to generalize beyond the* 

study. The sub-heading of the article gerferalizes to a ^^narsorjatch between • 

the cognitive level of college students and the content and teaching of 

college courses." This may be misleading to' the less than critical reader 

for the present "study .did not focus on the content and teaching of college 

courses. His conclu?5ion that "the '^jority of hipji school and college 

students' are below the formal level in^cognitive functioning" generalizes 

to a population beyond that Represented by the subjects of the present study. 

** 

The Tionclusion that "teachers should b^ aware of their students' levels of 
func^tioning and plan educational experiences accordin?>ly" is certainly con- 
sistent with accepted principles of educational psychology. ^Yet if, as 
Flavell 0977) argues, hypotho ti co-dcduct>i ve reasoning and scientific reasonr 
ing are synonymou':; , should not advanced courses for science ma jors require 
formal' reasoning abilities? 



0 
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In.summary^^olodiy ' s study avoids a' major criticisn of many studies in 
education, the lack-of a , theoretical basis. His ^^iiidings are p.cnerally 0 
consistent with and contribute to 'those of a rather large nunber of con- 
temporary studies based on.Piaget's theory of cognitivQ development at 
the formal-operational level (Chiapetta 1976). Unfortunately, afe-des- ' 
ctibed by the journal article *that serves as the basis for this ana'lysis, 
the present study makes no new methodological contribution to Piagetian 
research. Nor. does it provide a good mode}, for other investigators j^ecause^ 
of the many omi*ssions that have been cited which make replication problem- 
atic. Generalizing beyond the population represented by the subjects of 
the study is unfortunate dn that these generalizations are often J^emembered 
and cited by the less-than-critical reader. * . 

The difficulty in establishing equivalency of groups of subje'cts points to 
a rfeed for longitudinal studies to determine v/hat changes, if any, occur 
in mental level from high school through college. 
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Purpose 

The primary purpose was to assess the understanding of concrete and formal 
operational conc^epts by concrete and formal operational students in .second- 
ary school science classes. It was generally expected that a relationship 
would be found between students' scores on selected Piagetian tasks and 
written' tests measuring concrete and formal operational science concepts. 
That is: the concrete operational students would be able to understand 
only concrete concepts while formal operational students would be able to 
understand both concrete and formal operational concept5S. 

\ 

Rationale 

The study relied on conceptual differences between concrete oper^ional con- 
cepts and formal operational concepts. In the study, concrete operational 
concepts were defined as "concepts whose meanings can be developed from 
firsthand experience with objects or events", while formal operational con- 
cepts was defined as "concfepts whose meaning is derived through ^(Ss^tion 
within a postulatory-deductive system." Hence, the study assumes tkat a 
concrete concept may arise from direct sensory and tactile experience while 
formal operational concepts arise in a formally or logical deductive manner 
through imagination or through their logical relationships of ideas within 
a system. 

« 

As is comjnoh ,ln ^tudicy^ of tills type, emphasis was placed upon understand - 
ing formal \jKirational concepts, rather than upon knowing them. Understand- 
ing In this study was limited to students being able 'to answer comprehension 
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and application type questions'^bascd on Bloom's Taxonomy. Undoubted!^ the 
^ study is"dra\rm from other earlier studios in the general area of Piagetiah 
research^ Jiowever, the authors did not provide any clues as t;.o which, studies 
in this^^yast area of research were of particular importance,* The basic, 
reason for the study was fo question the validity of contemporsry high 
school -science curricula that present fornal operational *type concep"^ to 
a population that is primarily concrete dperat::|.onal , * 



Research Design and Procedure 

The design might he b^est described as an ex post^-facto type design resem- 
bling more an after-the-fact assessment without an/ real treatment control. 
Measur:es were constructed and administered external to all classroom instruc- 
tion. This included content tests as well ^s. Piagetian tasks. The four 
Piagetiah tasks included: conservation of weight, conservat;jion of volume, 
separation of variables, and equilibrium in the, balance, , In scoring the 
Piagetian tasks numerical scores were awarded on a 1 t^ 5 scale with pre- 
operational being 1 point, transition to concrete 2 points, concrete 3 ^ 
points, transition to formal A points and formal 5 points. Subject matter 
measures consisted of 15 concrete multiple choice questions and 15 formal 
multiple choice questions. Each set evaluated major -concepts taught during 
the school year;^ Teachers did review the questions to check the representa- 
tiveness of the subject matter included; this was identified as "content 
validity/' A Spearman -Brown split correlation was used* to estimate reli- 
ability of the subject matter tests. The Ss^ in this study were selected 
from one high school of over 200O students and represent a generally above 
average, suburban population. 



A number of statistical manipulations were used to lassess. the relationship 
between performance on Piagetian tasks and performance on tests. Included 
was a semlpartial regression to assess which Piagetian task 'best predicted 
the Ss_^ scores on the concrete and formal operational portions of the 
written examinations. Although Ss_ were scored on a 1-5 basis on the 
Piagetian tasks^ they were somehow assigned to one of seven levels of 
operational levels. It^is not clear how, this assignment was related to 
performance on the Piagetian tasks. Ge nerally, data were summarized in 



terms of pefcen.tages of correctly nnscwered concrete question-s and percent- 
ages of^. correctly answered formal concept questions. In addition, a step- 
wise multi^l^e regression 'table was included summarizing the data for the 
best pr^ictor!. 



Finding^ ^ ' ' ^ 

Of the biology sample, .about two-thitds of the Ss_ were categorized as con- ^ 
> 

Crete operational or posj: -concrete ope^tional. In the chemistry sample, 
most ol the were' in transition between concrete operational and formal 
operational with alinpst all the sample caregorized as post-concrete formal 
operational and transition to formal. In the physics class, S_s were 
generally between concrete operational an^ formal operational. However, 
the physics sample exhibited a flat distribution wi^h Ss L6cated in seven 
categories . ^ ^ , . " ' , 

, Concerning the relationships between students and their performance on the 
content ^rea questions, most students in the lower lev^^s of intellectual 
development found^the tests particularly dif^f icult -while students in the 
higher levels found the *tests easier as exhibited by the percentages of 
correctly answered questions. This pattern held true*for concrete concept 
questions as well as formal concept questions. It was not at all surpris- 
ing that all students found the formal concept questions more difficult than 
the concrete concept questions. Multiple correlation coefficients, which 
were not reported, were significant at the ,05 and ,01 levels, .(One must 
assume) that the authors mean significantly greater than zero. It is 



entirely possible that these may be significantly greater than zero but not 
account for much variance.) These unreported multiple correlation coeffi- 
Vients were summarized to suggest that Piagetian tasks are significantly 
correlated with understanding as operational as cTeTlned by the investiga- 
tors and that there appeared .to be more positive correlation between the 
^;^-H.3sks and their understanding of formal concepts and concrete concepts. 
The senipartial regression analysis disclosed that the conservation of 
volume task and the separation of variables task were the best predictors. 
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Inl^rprt nat ions \ . 



Generally the data were 'interpreted the authors to support the premise 
that concrete operational _S£*were unable t© develop understanding of formal 
concepts. Also, concrete operc^X ionafl S£ were able to denvonstrate under-' 
stariding of concrete concepts^ Formal operational i>ubjects were considered 
to be able to develop .ufiderstanding Of both concrete and formal .concepts.. 
Additional interpretation of findings suggested that the '^apparent retarded", 
development in the biology class may be traced to' inappropriate subject 
matter and teaching procedures, In general, the authors suggest that- a sub- 
stantial portion of secondary ,^^chool subject matter may not be suitable in 
terras of intellectual level of the learner, especially in.the case <of 
biology studei)ts' who were unable to develop appropriate understanding of 
formal abstract concep.ts. Ztk would appear that a^ science course dealing 
with abstraction and bas|c qtiifying themes is inappropriate, 



-ABSTRACTOR'S ANALYSIS ^ 

- 

i t 

} ^ 

Little new information wgg provided by the study. It is not surprising that 

♦ I, ' * * 

the students had difficulty answering difficult questions. Higher level 

quest ions .(Blood's Taxoxtomy ) are more difficult for students to answer 

because they requirexmore . information for the student to think about in 

; ; ' * ^ . ^ , 

order to answer. It^w^s not surprising that students or\i a concrete level 

' . \ ' * 

find simpler questions e.asier to answer. 

Many specific problems e^ist in 'the study: The statistical analysis^, the 

,<i*ef init ion of concrete-\operational concepts, the method of assessing under-- 

• ' I' • * 

standing, the lack of' protocols for Piagetian tasks, and so forth. Atten- 

A 

' tion should be given to research which attempts to 'link classroom achievement 
an.d instructional techniques to in tell e^tual development. However, in 
constructing studies of this nature, comparisons should be made that will 
help teachers and curriculvim 'developers modify the materials that they are 
using or producing. Certainly it is unreasonable to ask the curriculum 
developer to leave out all that "theoretical stuff of science." In many 
cases, the theoretical-slufX is the backbone Of science. Science must be 

" o • • ■ . 
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viewed as more than a collection of hands-on experiences; it inu.4t include * 
the development of abilities to construct explanations. The concern of this 
type of research should not be to find out xvliat concepts students are no£ 
understanding biit rather to find out how students can be helped to learn 
those concepts. » ^ 

* 

Specifically, the following are some directions for research of issues in 
this study: 

1, View carefully the methods of instruction in relationship to the 



learner's ability and level of development. In this study, no ^ 
attention was given to the ^style and instruction. 



2. Provide careful descriptions as to how data are managed and how 
Ss^-are assigned to groups. The res-earcher is obligated to explain 
his statistical, scoring, and other data management and collection 
activities, 

i 

3. , Select prediction measures (tasks and other assessments) which . 

fairly represent the operations or abilities necessary in learn- 
ing the content. Then, remember that they 'relate to the learning, 
not always to the testing. What is known in this study is that 
some learners on concrete levels had a great deal of difficulty 
completing formal level ^test questions. Was the content also'- 
presented in a formal level? We;re the learners unable to ] earn 
the concepts or only cpmplete the test? 

4. Treatment design and methodology need not be a problem. Much has 
been written in the area 'of Aptitucfl-Treatmeht Interact ions ^(ATI)^* 
Many of the problems not solved in this stu^y have been addressed . 
in ATI literature. 

.5. Interpretation and recommendations must be reasonable in yiew of a^' 
study^s limitation and realities of the classrooms, content areas, 
and teacher's skills, 



\ 
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Sayre, Stev^f; Ball, Daniel W. *'Pin^ctian <lor.niLive Development and 

AchiGveiueni ''ino Science . " Journal o f R csoarch in Science Teachi np,, 
12(2):165-174, Aoril 1975. ^ ^ 

Descripto'i's — -Accfdem c' Achievement; -''Copnitjlve Development; 

^''iD(^veloi)iiiental Tasks; Educational Research; '^Learnin^ Theories; 

ProdicXorxVari ables : Science Education; Secondary Education; 

'^Seconddry School 'Scitince • 

Expanded 'Abstract and Analysis Prepared Especially for I.S.E. by David 
J, Harmon, Th^ Duval Coun^'ty School Bo^ard, ' Jacksonville, Florid 

■ . -J . / 

Pu rpo3^' V 

The stated ^purpQ'se of the^^tudy was to determine the relationships between 
scholastic grades in s^ence of junior and senior high students* and the 
ability of students/^o perforin formal operational tasks. There wjai 
hypotheses test^ 
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Rationale 

According to Piageti^n theory, intellectual development evolves through 
four stages:^ (1) the Sensorimotor Stage (0-2 years), (2) the PreopeYa- 
tioj it^r Stage (2-7 years), (3) the Concrete Operational Stage (7-11 years), 
and (A) tl^e Forttjal Operational Stage (il-l6 years);. It was assumed that 
*the stages are of invariant .sequence and may vary in duration and chron- 
ologi(^al age .\.^Empirical support for purporting a relation^ir^p between^ 
cognitive devel^ment and achievement (science or in other disciplines) 
was not included. Elaboration of possible relationships between cogni- 
tive . development and learning could provide implications for the science 
teacher and the teacher educator. 



R esea^rch Desir;n and Procedure 

Four hundred nineteen science st^udents (214, grades 7-9, and 205-, grades 
10-12) enrolled fn eiglit Weld County (CoJorado) Rcorganizctl School Dis- 
trlcts were randomly selected to- participate in the study. SjLudents 
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successfully -completing four on more of tiie five criterion tasks of the 
Piagetian Task Instrum ent (PTI) were classified as" 'formal operational; 
others were classified as nonformal onernnional. 

Ppiir interviewers were trained to promote' oroper administration and 
evaluation of^student pei;formance on the PTI. Tnterviewer reliability 
. (percent o^£ agreement among interviewers) was established during a pilot 
sfc^icTy wi\h 16 ninth grade students. Tlie percent of. agreement was 81 ox 
higl^^r for student performance on» each t,3sk and. overall. Face validity 
was opined by^using tasks modified from Inhelde^ ^nd Piaget| (1971) , 
and KarpjAis and Peterson. C1970) . The reader should note^\he Ihhelder 
and Piaget\(1971) referenc.e wab cited but not included 'in the biblio- 
graphy. <The five tasks, are listed, relevant equipment is iderfttfied 
and the interviewers' and students' roles are discussed below. 

Task 1: .Stickmen -(Prepositional Logic and Hypothetical Deductive Reaso n- 
ing ) . ^ The S^ Vas j^iven "an 8% by 11 cardVith two stickmen drawr^ one on 
each side of tlie' card. The stickanen varied ^ip "height, one being two- 
^third^^ the 'height of .the other, S_ was a^ked to measure a^d ^record 

the height of the two stickmen with a set of, ei^ht connected '''jumbo" 
aper cj.ips. The "j umbo"^ paper clips were replaced with a smaller set 
of 12 paper clios. Jhe S was then asked to measure only theWshorter 
stickman and predict the Heigiit of -the larger stickman. » ' 

Task 2: Pendulum (Combinational Lo^ic and Hypothetical Deductive R easo iv- 
ing) . Tlie £ was given a -sinple pendulum consisting' of^ string, which 
could be shortened or lengthened, and a set of varying weights and asked 
to determine which .variable(s) affect (s) the f reqtiency of pendulum 
Oscillation. The problem was to isolate ^*%tring length from other varia"- 
'bles. ' ^ , . . ^ 

Task 3: Balance (Proposi tional Log ic and Hypothetical Deductit^e Reas on- 
ing ) . The was presente3 af balanced scale. One weig>it ^^as removed wUile 
the interviewer maintained ex]uilibrium by holding tlie forcS^rm^ . The S 
was asked, "Using any of the weights in front of you, hc^J^ could you get 
the scale to balance?" Aft^r the S's respons^^e, another(^ques tion v;as 
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^s^d," ^'Are there bthct ways to balance the scale besides the one yoi; 

chose?" %e intetviewer ,then removeH the we.ight from 'the scale and 

placed another Wei^it nearer 'tlie fulcrum wiiile maintaining equilibrium 

hy holding the. force arm. The S_ was asked how he mi<>ht balance the 

^cale/ and to jus^tify his responses. ' " . 

\ . i . ^ 



Task '4: CheTuicals (Combinational Lop;ic and Hypothetical Deducttve 
Reasoning ) . The S_ was given four flasks containing perceptuhlly iden- 
tical liquids. Tlie flasks (numbered 1-4) contained:' dilute' 'sulfuric 
acid, water, hydrogen peroxide, an(J- thiosul fate . A fifth flasl<^on- 
taining iodine was pre^^ed and labeled 5. The was given twp glassesj 
one containing 1 + 3, the other containing 2. The contents were not 
revealed to the S. Several drops of. were poured into each of the two 
glasses. AftQr observing the reactions, the S_ was asked, ."How can you 
reproduce the color?"" Tlie. S was allowed to attempt to reproducSg^,^hfe 
color. If successful, the S was asked to ''identify the functions oftHe^ 
liquids. , • ' ' ^ 

Task 5: » Syllogisms (Proposit ional Logic and Hypothetical Deductiv e 
Reasoning) . Three syllogisms were plresented to the S^^each consisting 
of two premises and a conclusion. After' Che S_ responded that the argu- 
ment was valid or invalid, he was a^ed to justify his choice* 



Scholastic grades wfere obtained *for both formal and i^onformal opera^ 
t ional Ss^/at each of the six grade levels. Data ^ere analyzed using 
statistical techniques, including Chi-Square Test of Independence and 
Point Biserial Correlation. A controversy as to whether scholastic 
grades are categorical or continuous data prompted the us^ of these 
two techniques. - ' ^ . 



Findings 



i 



Both 'formal level junior and senior high Sj5^ received sigiii f icantly higher 
(p^O.Ol) grades than did nonfornvil Ss. In regard to tested hypotheses: 



1. There was a significant relationship (p <. 0.01) between scholastic 
success of eighth grade, ninth grade, biology and chomis'try students 
and their performance on the KIl. • . - 

2*, There wa^ no significant relationship (p < 0.05) btstween- scholastic 
success "Of physics and seventh grade science studeijts and their ' 
'performance on the^PTI. 

Significant x:cfrre^ations (p>< 0.01) were found • between the number of 
tasks performed at the 'formal operational level and science 'g^:a<les of 
^juniQr high SsUr = 0.33), and senior high Ss^ (r = 0.A6). A significant 
relationship (p\o.05) resulted between tl;e scholastic science grades ^ 
of both^junibr and\enior h\igH _Ss and their performance on each of the 
tasks. 



Interpretations \ " % * ^ 



Students defined as formal operational received higher scholastic grades 
than did students defined as nonfoimial. Sayre and Ball state '\ . . the 
findings seem to indicate a gradual Wowth of science ^tudents" to complete 
formal^ operational tasks." They maintNain that this study reaffirms^ 
Piaget ' s. conceptions of the developmental growth of tke intellect. 

It was recommended that: (1) preservice teachers should develop greater* 
understanding of Piagetian theonCw'^ and (2) Secondary science instruction 
should be structured auround the cognitive de^^plopmental levels of the 
students involved. 



' ' * ABSTRACTOR'S ANALYSIS 

Pilot Stydy 



\ 



The 'ajuthors contend that interviewer reliability for the PTI was esta- 
blished during a p\ilot study. The pilot sample was limited to 16 ninth 
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grade students euroj.led in the Un j versity •of'^rthern Colorado Laboratory 

School. The reader does not know: (i) the similarity of the ninth grade 
•* ^ « . 

students to t'he main study sample (grades 7-^12); or (2). if the pcpfcent of 
agreement among interviewers would be. equally acceptable if grade levels 
C7-12) had been represented. • » * . 



The Popu!tatri!on and Sam ples 

Two random samples of science students were selected, one at the junior 
high level (N = 214) , the other at the senior high level (N = 205) . 
Simple random sampl-ing can only guarantee representativeness in terms of 
equality of probability for selection. This procedure does not guaran- 
tee inclusion of Ss^ from any particular subgroup of the population (e g., 
grade levels) . A stratified random sampling technique would be needed 
to attain grade level representation, ^^veral unanswered quesTions 
remain: (1) How was the sample size del^rminedf (2) How^ man^T teacliers 
and schools were represented by ^he selected Ss ? (3) Were cultViral 
differences that may affect cognitive development adequately co^itrolled? 
A more elaborate pilot study would have provided implications for ques- 
tions 1 and 3. Other comments^related to sampling are mentionecj in the ^ 
next section . ^ • , ^ r 



Variables 

The authors did not specify whether quarter, semester, or year science 
grades were employed. Nor did they provide an interviaw timetable. 
It is likely that internal validity was threatened by maturation. 

Level of Cognitive development. The method used to classify subjects 
may be open to question for several reasons: 

^1. Lunzer (1965) f ouiid^^jjiat Ss_ who perform a task at one level of 
thinking may well perform fhe next task at a moYe or less 
sophisticated level, even if the two tasks are logically 
similar.. 
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2. Beard (1962^ founcj that for 8 to 16 year olds, the level of 
logical' tliinking varied extensively between schools. As 
with the study being, reviewed , the ability to control varia- 
bles was included., ^ 

*^ 

3. Linn and Levine (19'76) found that adolescents' (ages 11-16) 
success rates on solving 'logical problems depended on the 
familiarity of the variables. 

4. No ration^ire was presented for using successful completion 
of four or five tasks as a cut-off point".' 



Perhaps developing a more specific scoring procedure and comparing. 

V 

student responses 6n logipally similar tasks would have' strengthened 
the authors' classification method. 



Science Grades, This abstractor finds it difficult to think of a measure* 
less reliable than scholastic grades, especially when ^collapsed within 
and over grade levels^ A list of possible extraneous fact/^s would be 
lengthy.* Concerning grades, Schwebel (1972) noted that they are probably 
based on the ability to .memorize, particularly if a nonformal student is 
confronted with formal stibject matter. * 



Poindings It 

The statisj^ical techniques seem appropriate. Many of the^ findings were 
presented by school types; perhaps an hypothesis should have been^^tated 
for each (i.e., junior and senior high). 



Finding £ (page 171) regarding junior high science Ss_ is not supported 
by' the data. A significant (p<0.05) relationship was reported between 
the scholastic science grades of junior high students* and their perfor- 
mance on each of the tasks. Table IV (page 170) indicates no significant 
relationship between scholastic grades of junior high students and their 
performance .on either Task 4 or 5. Data-in Table V (page 170) are less 
supportive. ' * * / 



With -junior {dgh students, finding a_ (page 170) indicates a correlation, 
of .33 betweeg number of tasks performed af the formal level and grades. 
Table IV (page^l70) indicates that r = .38. 



No significant relationship) (p<-0.5) was obtained betwe\^n gVadfes ^of 
physics and seventh grade science Sg_ ahd^ performance on tlie PTI.* In, ' 
explanation, it was noted that 91. A petcent of a^Venth grade Ss w<^re 
classified as non^ormal and 80.7 percent of the p^^iy^s Ss were consid- 
ered formal operational. The .strength of this explanation is j^&4^ce;j4)y 
the following^ ^ (1) Table VI (page 172) indic^es th^t 90 percent of 
eighth and 86.5 percent of ninth grade vere "classified as nonformal- 
and (2) 89.3 percent (abstractor's calculation) .of all junior high Ss 
were nonformal. Perhaps a closer look at physics and seventh grade Ss 
is warranted^ y 



Summai?y , ^ 

»■ • 
This_ -research attempts to bridge the gap between theory and practice. It 
is likQly thAC the study has stimulated the interest of teachers in the 
possible relationships of individual differerices and ach'ieveraen t . 
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Purpose , ' , 

The investigation was* designed to study the following questions: 

(1) Can the ability to control variables be taught to high scho 
biology studep"ts who do not demonstrate/ formal reasoning on a written 
test of logical* operations? , 

(2> Are 'students who are classified as early formal operational 
thinker^ on. the wi;atteu,.test of logical operations able to benefit more 
fro.m^he tVaining .than students v/Vio are .classified as early or late con- 
crete oper;ational think^rg? ' ^ \ ^^^""^-^^ 

(3) '^f the atilit^/ t6 control 'variables can be learned, is it 
geneyalizabla to problems ^utiliEing noyel^ materials? 



Rationale ^- 

^ I . - ^ • 

The investigatAort 'was, conducted using the Piagqtian model of. intellectual 
devel9pment as^ its theoretical framework. 'The .inve3tigation represent?ed 
an attempt tq test P-lag^t's position with tegard to the acquisition of 
forftial reasoning ability; .It was related, tp earlier investigations by 
Case (1974), Bredderman^ (1972), Bass and 'Montague , (1972) and Case and Fry 
(1973).. The Piagetiart tasks and\5coring procedures Aise<i in the inVesfei- 
gatidn were pres^ted in detail in an earlier study by Lawson, Nordland 
an^ Devi to (1974). f ' ' 

the lU^or assumptions underlying the investigation were the acceptance 
pf the position diiat the ability to control variables requires formal 



reasoning ability and the acceptance of the Longeot pencil arid paper 
test of logical operations as a valid and reliable means for classifying 
students into selected Piagetian levels' of intellectual development. 



Research Design and Procedure 

Tlie researchers reported that a posttest-orfly design was used i^ the 
study. Sixty-five* high school biology students whose ages ranged from 
14 years, 7 months* to 17 years, 10 months were randomly assigned to an 
experimental or a control group. The 33 students in the experimental 
gfoup received training in the ability to control variables while the 
32 students in the control group received no training. 

Students in the experimental group were further subdivided into two groups 
prior to ahe training sessions. Students in each of the experimental sub- 
groups worked in pairs and participated in four, 50-minute training 
sessions which followed the SCIS exploration- invent ion-discovery sequence 
and mode^ of instruction. Two pro^ems- involving the control of variables 
were used during the training sessions. The problems included the deter- < 
mination of the period of a pendulum and an investigation of variables 
relafed to the number of rotations of a rotoplane. During the first 
session, students participated in an exploration ;j^e,sson using the pendulum. 
The concepts of variables and period of- a pendulum were introduced in the 
second session. The concept of a controlled experiment using the pendulum 
was introduced in the third session. The fourth session was a discovery 
lesson. It provided the opportunity to apply the concepts of variables 
and controlled variables' using a rotoplane. 

The posttest consisted of interviewing each student in the control and 
experimental groups on three Pi|agetian tasks; i.e., exclusion of irrele- 
vant variables, s(iparation of variables and equilibrium in the balance. 

Students in both the experimental and control groups were given the 
Longeot pencil and paper test of logical operati^ons prior to the training 
sessions. The test reportedly allowed the classification of the students 
into concrete transitional, and formal operational stages of development. 
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A t-test was used in analyzing differencos between experimental and 
group mean posttest scores on the three Piagetian tasks. 



Findings ' * 

The findings reported by the investigators were as follows: 

1. Student responses in both experimental and contr<ol groups were 
classified on the Longeot test of logical operations as being 14 percent 
early concrete operational (IIA), 4*1 percent fully concrete operational 
(IIB) , 35 percent^ early formal operational (IIIA) , and 8 percent fully 
formal operational (IIIB) ; . ' * 

2; The mean scores of the experimental group (15.2) and control group 
(14.9) on the Longeot test did not differ significantly at the 0.05 level; 

3. The mean score of the experimental group (3.12) on the exclusion 

task was significantly larger at the 0.01 level than the mean score of the 

^ 

control group (2.53): 

V- 

4. There was no significant difference be.tween the mean scores of jthe 
experimental group (2.57) and control group (2.56) on the separation of 
variables task; 

5. The mean score of the experimental group (2.57) did not differ 
significantly from the mean score of the control group (2.69) on the equil- 
ibrium in the balance ta*sk; and 

6. Students in the experimental group who were classified IIA, IIB or 
IIIA on^the Longeot test^ showed an overall average gain of 1.5, 0*69 and 
0.33 substages, respectively, when classified on the Piagetian tasks 
following training, , ^ • 



Interpre tat j ons . . 

Tli<^ investigators concluded tliat training can increase student pcrfoi*man(p.e 
on a task designed to measure the ability to control variables when the 
task involves materials similar to those used in the training. The fact 
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that the control grOup performed as well as the experimental group on 
the tasks measuring transfer of trctining suggested that the improved 
performance on the trained tasks was the result of rote learning^rather 
than an increase in intellectual' development* 
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Another stated conclusion was that ^students who were class^if ied" as' earTy ' 
formal operational on the Longeot test did not benefit more from the 
training than students wJio were classified as concrete operational. 

Upon analyzing why the training* was not more successful, the investigators 
decided that the factor most likely missing from the training sessions was 
the self-re gulatipn or equilibrat iop factor. They pi?oposed a hypothetical 
model of how self-regulatipn might operate in the training sessions and 
predicted that students would perform at an increased level if they parti- 
cipated in more experiences similar to those in the training sessions over 
a longer period of time and if^they were encouraged^to think through the 
problems at their own individual rates. 



ABSTRACTOR'S ANALYSIS 

Ove^ the past 15 years, numerous researchers have examined whether or not 
training can result in the acceleration ^f students through the Piagetian 
levels of intellectual development.^ An analysis of the<se investigations 
have revealed that students can be taught to verbalize or perform specific 
tasks requiring higher developmental levels, especially when the verbali- 
zations or tasTcs were similar to those introduced in the training sessions, 
but the ability t6 perform at higher levels was not generally retained or 
trartsferable to novel situations. A portion of the results of this invest- 
igation by Lawson, Blake and .Nordland. support the conclusions draim from 
earlier investigations. It does, however, provide the added dimension of 
further supporting the .notion that improved performance could be attri- 
buted to rote learning rather than an increase in intellectual development. 
Collectively, these investigations uphold the Piagetian prediction regard- 
ing acceleration of development of students from one intellectual stage to 
another. 
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A second finding reported by Lawson, Blake and Norddand did not uphold 
what the investigators stated a*s a Piagetian pi'Gdiction. 'Hie prediction 
being that training would only be effective for students classified as 
transitional, defined In the study as late concrete operational and 
early formal operational. Students classified as early concrete and 
fully concre.te operational showed greater improvement as a result of the 
* trainin-g^than did students cla'Ssifled" a-^ early formal x)pBrationBl . The 

investigators were quite surprised by this finding and "Suggested that it > 
may have been the result of rote learning and ceiling effect. An addi- 
tional' rival hypothesis for explaining the finding and one not mentioned 
by the investigators may have resided in the quality of the pre- and post- 
test instrunients and/or the interrelationship between what the instruments 
were purported to have measured. Unfortunately, a detailed description of 
the instruments, including discussions of their validity, reliability and 
intercorrelations were not. included in the' research report. The only 
bibliographical references made regarding the Longeot test were incorrectly 
listed and therefore unavailable. 

The research report would have been improved if it had included additional 
information regarding the testing instruments. Some information on the 
tests^ was available, nevertheless, in a more recent report by Lawson and 
Blake (19^6) , They made comparisons amon^ three tests,' including the ^ 
'three Piagetian tasks, a biology examination, and a 19-item ^rsion of 
the Longeot test. Chi-square analysis yielded a significant relationship 
at the 0.02 i^veT^tueen student classification on each Piagetian level 
. on the Longeot test and three Piagetian tasks. The results of the chi- 

square analysis presented by the investigators appeared impressive, but 
Liglit (1973) has pointed out* that chi-square suffers from a lack of inter- 
pretability. It^ is^ possible to obtain a highly significant chi-square 
value even though the variables are only slightly associated. The chi- 
square value showed that a pattern existed between the test results that 
was significantly different from a random pattern, but it did not provjLde 
a measure of the correlation betv/een student placement on each Piagetian 
level by the two instruments. 

In fact, data from the contingency table revealed a skewed pattern with 
students being* placed on the same Piagetian level on both instruments 
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only 29 out of 60'^W!hes or roughly 48 percent of. the time. Students 
were classified at higher Piagetian levels on the Piagetian tasks than 
on the LongGOt test 23 out of 60 times or approximately 38 percent of 
the time, ^This apparent skewed relationship suggests a plausible explan- 
ation for the concrete operational students showing greater improvements 
than students classified as transirional or early formal operational as 
a result of the training sessions, Scbdents classified as fully concrete 
operational on the Longeot test may have actually been transitional while 
tho'se classified as being transitional on thX Longeot test may have been 
fully formal operational as measured by the Piagetian tasks. Such an 
analysis reveals the necessity for an indepth exai^nation of the testing 
instruments. It further suggests- that the discussion by Light (1973) 
on the analysis ^of qualitative data in the Second Handbook of Research 
on Teaching be examined before accepting chi-square as a method for 
analyzing data in contingency tables. 

One question should be raised regarding the chi-square analysis between 
student classification on the Longeot test and three Piagetian tasks 
beyond it being a questionable method of analysis. A recalculation by 
the reviewer revealed a chd-square value somewhat less than the reported 
value, Tlie recalculated value was significant at the 0.11 level rather 
than the 0.02 level. 




In addition to the 'investigation by Lawson and Blake (1976), the Longeot 
pencil and paper test of logical operations was described in a companion 
research report t)y Lawson (1975) and a more recent report by Lawson and 
Wollman (1976). The original test contained ^8 items, but a 'shortened 
form was, used in the previously identified reports. Lawson (1975) admin- 
istered a 15-itei>i^version of the test, although he stated that he had 
.reduced the .test to 14 items. Lawson and Blak^ (1976) used a 19-item 
version of the test* and Lawson and Wollman (1976>4^hort ened the test to 
8 items. The use of the inconsistent number of items in these investiga- 
tions and failure to identify which versioi^of the test was\jsed in the 
present investigation provides a very confusing situation. InXaddition^ 
it is diff^icult to have confidence in a test when such procedure; 
coupled with expressing the reliability in terms of internal consistency 
and not re-establisliing validity, are used. Modifying can and frequently 
does have serious effects ^on test reliability and validity. 



An explanation of why the research dosigil was neferfed to as a posttest . 
only design also would have Improved the research report.. Experimental 
and control group posi:tests were used fox answering the first .question, 
but the Longeot test apparently served'as a pretest" and three Piagetian 
taiskg were used as a posttest»for answering the second question of which 
level of students benef ittec}' most from the training. The Longeot test 
was not called a pretest; however, it was used^ to classify students into 
the same Piagetian levels as did the three Piagetian tasks; 

Ihe recognition and statement by the investigators that the ^apprbpriate- 
ness pind/or quality of the training sessions could have been the reason 
for the lack of success or the posttests by the experimental , grouj) was 
commendable* The writers of other research report-s would do well yd 
''\follow this example when it is applicable. 

Tlie prediction that the investigators generated from their hypothetical 
model of how self-regulation might operate in the training sessions was 
interesting and congruent with the Piagetian model of intellectual 
development, Of co.urse, it remains as a proposition requiring further 
research. Additional areas of research include determining the test- ' 
retest reliability and establishing the validity of modified versions 
of the Longeot test, comparing various methods of analysis of data in 
contingency tables, determining whether or not understanding of the con- 
cept of controning variables i^ characteristic af high school biology 
students and examining different strategies for training students to 
control variables. 
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Expanded Abstract and Analysis Prepared Especially for. l.S.t. by U 
R. Brown, Old Dominion University. 

Purpose ^ • 



The purpose of this investigation was to determine the effectiveness of 
a child-structured science program in further developing selected intellec- 
tual factors that appear to •be sequent iaifey developed among pre-operat ional 
and concrete-operational children. A major component of the study xjas the 
deveiLapment of tasks to measure the hypothesized intellectual factors. 



Rationale » x 



The investigators proposed that instructional pro.grams should be g^valuated 
in order feo determine the effectiveness of the materials anji procedures in 
attaining stated goals. Research findings suggest that Guilford's 
stifucture-of-iill'fellect model presents the possibility <jf isolating specific 
intellectual skills that can^ be' further developed by specific learning 
experiences I^'^iKie experiences should be geared to the cognitive capaci 
and mode of functioning of children as described by Piaget. 

" . ' ' - ■ ■ . ' - 

Research Design and Procedure * . , 

The \^ariate was student experiences in a child-structured science program. 
Child-str.uctured was .^operational! y define,d as having thp follot/ing charac- 
teristics: (a) Mani^aoj^Iative materials .were available to every student 
during each lesson in quantities such that sharing w^.s no| required; (b) 
Sets of material.4 allowed for a variety of activities which^were deter- 
mined by the students; (c) The teacher avoided evaluation and directive- 
behaviors and attempts to interact with individuals. 



Intellectual factors hypothcbized to be strongly associated with selected 
sets of science mterials were: cognition^ conve^ent production, and L 
evaluation of figural classes: cognition and evaluation of figural rela- 
tions; cognition and converger 
evaluation of figural systems, 



tions; cognition and convergent production of tigiiral implications; ^and 



The criterion variable was performance on tasks designed ^r the study > 
that measured the eight hypoth'esized^intellectual factors. Factors in 
the test battery had a high probability of being develop'ed as suggested 
by the nature of the'^activities. There had to be as many task variables 
in the test battery as were needed to identify and qualify the factory 
in the factor matrix. The tasks and the respective factors ar& discussed 
in detail in the report. ' ■ ' , 

Two tasks were used only to indicate whether a child was^t the pre- 
operational stage in the development pf class concept. /A ^third task 
wW used to determine whether a child was- at the pre-operational or 

concrete-operational stage' of class concept develo^ent. • . • 

• ♦ *^ 

'"^^ ^ 
For size, area, and volume relationships, three tasks were /lesigned, to 

designate a child as being in the -pre-operational stage. ^ A fourtrh task 
assessed whether a child had passed into the concrete-operatioAal stage. 
Another task involved the evaluation of serial systems and*could not be 
used for st;age categorization. The final task was uSed to indicate 
whether the child was in the concrete-operational stage of the develop- 
ment of the concepts of area and volume-, respectively, or in the stage ^ 
preceding it. • . ' . ' 

A reliability estimate of 0.927 was oKtaineS for the cognitive test ^ 
instrument by the use of alternate components* 

A delta index was de^veloped for use in^ validation. If 'the rate of 
development of the intellectual factors varies, scores of -children' 
within a particular age group on items homogeneously measuring the same 
factor would be expected to cluster on* a development continuum. There 
would be as man^^lustcrs as factors being measured. 
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Task analysis was conducted to identify the tasks that contributed 

notably to the significant differences among the total test means of 

the subjects in each of the four treat ment-a^ie level combinations By 

analyzing for effects that could be attributed to treatment, age, or 

V. 

treatment-age interaction. 

^• 

The treatment group was 23 firVt grade stude.njLS. ^ Two control groups 
consisted' of 23 kindergarten students and 27 first graders. Intact 
classrooms were used. Age and socioec^onbmic status were used to screen 
students to be included in the project. The design model was treatment- 
posttest for the one class and posttest only for the control groups. 
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Toward the end of the school year students were interviewed on the tasks 
and responses were tape-recorded and scored. 

Where the means on a given task of at least two of the four age-treatment 
groups were below Ol90, a tv7o-way ANOVA was performed. t-Tests were used 
to compare means of all possible pairs of treatmet\t-age groups to deter- 
mine sources of significant differences. One-way Al^OVA was used to 
investigate ^e effect for those tasks where means of three of the four 
treatment-age^ groups were above 0.90. 



Findings 

All the children were observed to be in either the pre-operational or the 
concrete-operational stage. There was a significant difference (.01) 

between the treatment and the combined control groups, but age and 

treatment-age interaction w^s not significant. Results favored the 

treatment group in terms of the development of the concept of class, and 

size, area, arid volume relationship as well as the intellectual factors 
related t6\^thcse concepts. 
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Intdl'i>rQtaL ions • ' 

& 

The results support the conclusion that an activity-centered program, 
where the* experiences were designed to match the cognitive structures 
of children in certain stages of developmen^t , promotes ^the development 
of intellectual factors which enable children to move to the concrete- 
operational stage. 

Results of^he item analysis may be interpreted to imply tfiat the acti- 
vities on clarssif ication and size relationships promote 'the development 
• of the intellectual factors: convergent production of figural classes' 
an,d evaluation of figural systems, respectively. Cognition of figural > 
classes and evaluation of figural relations may be developed by thfe time 
children are about seven years of age without specific experiences such 
as those provided in a child-structured program. 

Intellectual factors from the Guilford model can be predicted, identi- 
fied, and assessed by using interview tasks and techniques derived from 
the work of Piaget. 

if 

ABSTRACTOR'S ANALYSIS 

This study dealt with the application of learning and cognitive develop- 
ment theories to classroom instruction. This has been indicated as a top 
priority for research by NARST members (Butts, 1977, p. 163). Tlie inte- 
gration of the Guilford and Piaget models is an excellent "start" at 
application of learning theories to science program planning. 

The investigators developed and validated a series of tasks linked to 
specified intellectual factors. The description of the tasks in the 
refTort would allow for replication of the study' The replicability 
potential of the study is an asset not found in maii'y educational ^'studies . 

' Prior to "t^lication or extension of the study, t^o points need to be 
craVified in tlie research desip;n. First the length of time of the 



treatment .should be specified. Tht^ invcstip,ators developed an excellent 
operational definition for a child-structured science pro^ran, but the 
reader is not informed as -to the duration of the treatment. Treatment 
length and related variables such as specific teaching stratep,ies used 
could be variables that affect 'performance on tasks. 
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A second design restriction is the lack of random assigninent of subje'cts 

to treatment and control situations. Randomization could certainly 

strengthen confidence in the results. ^ 

* • 

It would be helpful if the "delta index" referred to for use in valida-*^ 
tion were explained in a- si^mmary fashion. footnote specified that this 
information may be obtained from, the investigators. The conclusions 
reached are dependent, in part, on a valid assessment instrument. v 

Unfortunately, many "advocates" of Piagetian principles are enamored with 
age guidelines. The investigators in this study were careful in this ' 
respect and established two levels with 6:11 as the dividing line. ^ 
Reference to stages ratho^jr than ages lends credibility to the findings. 

i-" ■ ■' . \ 

A fundamental question of this study is what actually was evaluated? 
The investigators refer to "evaluation of a child-structured science 
curriculum." Their working definition of "curriculum" actually describes 
, instructional variables. If curriculum is "a plan, a structured series 
of intended student outcomes o-r a set of planned expetiences indicating 
that which has been identified as worthwhile for students to learn or 
'experience," then curriculum^ evaluation is accomplished tl\rough the con- 
sistent and logical use of statements in decision making (I^Jhi^, 1977, pp. 
3-6). The assessment of student learnings is a procedure for evaluating 
instruction. Instruction refers to the behavior of teachers^and the' 
strategies selected to promote the* objectives of the curri;^Uim. It* 
appears to this reviewer that the intellectual factors as identified by 
Guilford can be used in a curriculum^ plan. How "well" these factors are 
developed depends on instruction such as defined in/the tasks developed 
lor this study. 



^ ^le evidence gathered by ^this investigation supports the utility of the 

Intellectual model's of Guilford and Pia^;c»t in program evaluation for , 

' ■ / * ' 

young children. Development of tasks for older learners would certainly 

be a useful contribution to the research pool. 
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Expanded Abstract and Analysis Prepared Especially for I.S.E. by Anton E. 
Lawson," Arizona State University. 



Purpose 



The purpose of this study was to analyze the hierarchical scheme of seven 
operative structures defined by Jean Piaget as well> as determine if the 
sequence of two substages of concrete thought and two substages of formal 
thought cohstitute a hierarchy of related components. Three hypotheses 
were tested: 



1. Is there a hierg^rchicalyS^ructure among the concrete operational 
stages III A and B and the formal operational stages IV A and B? 



2. Is there a hierarchical structure among the logical operations 
of classification, seriation, logical multiplication, comp^ensa- 
tion, proportions, probability, and correlations? » 



Is there a factor describing stage III A operative structures and 
another factor describing stage III B, IV A, and IV B operative 
structures? * * 



Rationale - * / 

The operative structures defined by Piaget are important because invest i- 
gatioi\s have shown that they determine the form and function of 'concept 
acquisition. As an individual acquires more compl'^oc\)peratlve structures 
he Is capabl-e .of acquiring more complex concepts. 
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Piaget^s theory of development states that the development of more complex 
operative structures depends upon prior development of simpler operative 
structures. If the Bypothc^fRed liierarchy of operative structures of 
'classification, seriation, logical multiplication, compensation, propor- 
tionsf and correlations is confirmed then it means that the K-12 science 
curricula .should consideir this sequence in designing instructional sequences 
for science concepts. Conversely, if the sequence is not confirmed then 
science instruction need not consider the interlocking dependence of these 
structures. ^ 



0 





Research Design and Procedure 

Instrument — the Raven Test of Logical Operations contains 68 pencil-paper 
items testing for use of the seven operative structures investigated. The 
test is divided into three parts each requiring about 45 minutes for admin 
istration. 



Subjects were 896 male and female students ranging, in age from 8.8 years 
to 19.4 years. Subjects were selected from third, fourth, fifth, sixth, 
seventh, aad ninth grades ^d college freshman classes dra^m from* four 
metropolitan areas: Philadelphia, Pennsylvania; Buffalp, New York; Sala- 
manca. New York; and Atlanta, Georgia. 

Guttman's radex t;heory as modiried by tjchoeman was useT^s a q"uarvtitative 
technique to test the first two hypotheses. The third hypothesis was 
tested by use of Alpha Factor Analysis. T ' * - , 



Findings 



Hypothesis one was accepted in that the predicted order of complexity 
(stage III A, III IV A, IV B) was found. The rank order of the simplex 
loading coefficients associated with the substage levels was the same as 
the predicted order. 

Hypothesis two was rejected In that the rank order of the coefficients asso- 
ciated with the seven logical operation ^^^Hiables was not the same as the, 
p^rodJctod ordor. 
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Hypothesis three was accepted in that' tHe expected loading pattern of 
the ^even logical operations variables was observed. Classification, 
seriation, and compensation showed moderate loa^dingp. on factor II while 
logical multiplications, compensations, proportions, probability, and 
correlation showed moderate loadings on factar 1. ' 



Interpretations o. i 

The results of the "Study shqw that tiwp concrete sub stages and the two 
formal substages of cognitive development are hierarchically related but 
that some overlap exists. Presumably some operations develop in a parallel 
fashion. The seven logical operations^ do not by themselves forma linear 
hierarchy. This result suggests that several logical operations within a 

« 

group may be used to construcft science concepts ,by individuals at a spec- 
ific stage. The operative structures may be viewed as a means for 

> 

extending the use of any given science concept within a given level of* 
development thus enhancing the concept's generality and retention. 



ABSTRACTOR'S ANALYSIS 



This investigation was based primarily Qpon the rather" tenuous hypothesis » 

4 

that the seven operative structu^'es of classification, seriation, logical 
multiplication, ^compensation, proportion, probability, and porrelation 
form a sequential hierarchy in the order listed. Raven and Guerin admit- 
that Piaget himself does not suggest this hierarchy, nevertheless they 
claim that it is implied by Piagetian studies. ♦ 

Certainly some sequentiality must exist (e.g., classification and seriati£Jh 
must come before correlation) but I .find no justification , for drawing such 
an implication from the studies cited by the present authors. A more 
modest but much more believable interpretation of those studies is simply 
that tke logical operations of the concrete stage must precede those of 
the formal stage. This in no way implies that any two operations of^a 
single stage, such as proportions and probability, are^ sequentially acquire 
witliin a given stage. . . . 
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Raven and Guerin proceed fi*om this tenuous hypotl^esis to state that if 
the hypothesis is confirn^d, then the K-12 science curriculum should con- 
sider the sequence in teaching concepts. Further ^itey state that if the 
order is not confirmed, then science '^inst ruction need not consider the* 
interlocking dependence of these, structures. For example, no attention 
need be paid to compensations and seriat;ions when teachin/g concepts using 
proportions. I find this statement unwarranted and ifrijustified. To imply 
that the usefulness of Piaget's theory of the development of operative . 
structures rests upon the conf irmatiQji of the present hypothesis of 
sequen tiality is presumptuous. 



Neve rt;he less, the idea of a sequence of hierarchically related operative 
structures^ within stages is an interesting one. Unfortunately it is jiQ^ 
one so easily tested. The use of the factor analytic techniques employqd - 
by Raven and Guerin, although useful, cannot really tell us if,' say, 
classification precedes or follows seriation or if projportions precede 
or follow probability. This is becau'Se the analyses are dependent in part 
upon the difficulty level of the items. The most difficult items generally 
turn out to be the ones at the top of the hierarchy. But this need not 
imply that they develop last. This is simply because not all items involv-^ 
ing one operative structure are of equal difficulty. For instance, 'a number 
of items involving proportions and probability can be selected with a wide 
range of overlapping difficulty. If one selects only the easy proportions 
items and only the difficult probability items, then the statistical analy- 
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sis will show probability at the top of the hierarchy. If, on the other 
hand, only "feasy probability items and difficult proportions items are 
selected then the order will be rever$ed. So which comes first? The 
analysis cannot tell us. For this reason hypotheses such as this are 
extremely difficult, if not impossible, to test adequately. 

■ . / 

Raven and Guerin claim to have found support for their first hypothesis 
concerning the hierarchical &tructureof the concrete III A and III B and , 
the formal IV A and IV B substages. What they in fact found support for 
was the following sequence: classification and seriation (which tHey placed 
in substnge III A), logical* multiplication and compensatiQn (v/h Lch they 
placed 'in substage III B) , proportions and probability (placed in substagQ_^' 
*IV A), and correlations (placed in substage IV B). They have simply grouped 
test scores from* different parts of the test. Empirical support fot the* 



validity of the sequen tiality of this* grouping of items is found but at 
this point a quest ion -^s raised as to the value of such empirical support ♦ 

Let me clarify with an example. Take correlations, for instance* If one 
analyzes what is needed to solve a correTation task, it becomes imme- 
diately obvious that the data under question mus t be classified, then the 
number of instances of class members be_ compared, then joint probabilities 
or ratios of occurrences and nonoccurrences of class members must be com- 
puted. Without these initial steps the correlation task cannot be solved* 
Obviously then someone ^would not solve a correlation task if he were unable 
to solve the logically less complex task of classification and so on. 
This simply stands to reason » The empirical result that the correlation 
tasks ara more difficult and are solved only by those who can also solve 
classification tasks is not the least bit surprising. Other similar argu- 
ments could be made for the other operative structures investigated. 

> 

The 'results of the test of hypothesis three that showed the operative 
structures do contain two general factors support eai^lier factor analytic 
studies which indicate two general modes of thought: concrete and formal ♦ 
Raven a^d Guerin^s distinction between t;Jie concrete^ factor as one invtd^v- 
ing the arrangement of events and objects by property ^and the formal 
factor as one concerning the coordination of changing variables appears 
insightful and potentially helpful* However, the means for using these 
two operative structure domains for the impVovement of teaching scifence 
concepts sugges<:ed- by the authors is not at all made clear. 
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Purpose « ' ^ 

- " • * \ 

1. To determine relationships among students' scores on ^^r Piaget- 
ian styled tasks measuriag formal reasoning ability and scores on 

.measures of attitude, aptitude, achievement and knowledge of the 
science processes. ' 

2. Analysis of these interrelationshios to determine if the Piagetian**- 
styled tasks do reflect a meaningful measure .of the extent of for- 
ma], operational^ thinking abilities or if they simply measure 
physics or science content. ^ 



Rationale 

-* ' 

Recent studies indicate that as many -as -^0-^95 percerrt 0'f~^"^OTfer5^~aTi^ 
college-^evel stud^ts fail demonstrate formal reasoning ability as 
measured by Piagetian-styled tasks. The authors raise a question about 
the validity of these findings, suggesting the possibility that the tasks 
may be dominated by physics content which is sinply unfamiliar to students 



Research Design and Procedure 

An ex post f^oto design using hypotheses testing orocedures was employed. 

> 

Four Piagetian-styled tasks were u^ed td measure the formal reasoning 
abilities of 71 college fre?;hmen a^nd sophomore elementary education majors. 
The tasks and their reported reliability coefficient-are: The Conserva- 
tion of Volume Using/clay (.24), The Conservatjon of Volume Using Metal 
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Cylinders (.AS), ITie SeparadTon of Variables (,79) and The Exclusion of 
Irrelevant Variables (.79)L - / ' . 

Points were assigned to each task and subiects stratified into three 
categories based on the sum of their total scores, Tliese procedures 
identified 13 subjects as concrete operational, 47 as transitional; and 
11 as formal operational. 

The following achievement, aotitude", and attitude measures were obtained 
from student records and used as concomitant variables, 

A, Achievement measures ' * , 

1, Science - Biology/Chemistry College Entrance Examination ' 

2, Science - Sequential Test of Educational Progress 

3, Science - The Wisconsin Inventory of Science 

4, Mathematics - College Entrance Examination 

5, English - College Entrance Examination 

6, High School Graduating Class Rank 

College Gradepoint Average ' - 



B, Aptitudd measures ^ 

8, Mathematics - Scholastic'Aptitude Test 

9, Verbal Scholastic Aptitude Test 



C, Attitude measures 

10, Attitudes toward Science and Science Teachings- Tlie Bra-tt 'Attitude 
Test t ^ ^ 

-/ 

The data were analyzed in three ways:^ ^ 



1* Intercorrelatif>ns were run on the task scores and the total task 

f 

score with all achievement, aptitude, and atti tude .measures . 
Correlations which reached significance at the ,05 level were 
corrected Tor attentuation. 



ERIC 



43 



2. One-way analysis of variance procedures were used to test for 
significant differences airiOng the means of the ^oncrete, trans- 
itional, and formal operational groups on eachsof the dependent 
variables . 



Principal Ct)mpo'nents Analysis was run to determine the number of 
significant factors amonp, the variables measured. Only loadings 
greater than ,30 were included. 



7^ 



Findings 



- • « ( ' 

The majority, ^6 percent of the sample, demonstrated transitional responses 
With the exception of the conservation of Volume Using Clay task, results 
obtained on the Piagetian-styled interviews were consistent with previous 
studies. On' the Clay task> 90 percent of the~students exhibited conserva- 
tion teasqning compared to previously reported percentages of 58 by Elkind 
(1962) and ^1 by Towler and Wheatly(1971) . Excep^t: for this same task, all 
other task scores correlated positively with all of the other measures. 
The correlatioTTS among students' total task score* and all othA measures 
were found positive and si^ificant. ~" 

Correlations among the task scores and the science achievement tests as 

well as the mathematics and English achievement tests were^ significant, 

with the science measures somewhat higher. The correlations of task 

scores with the verbal and math aptitude scores were moderate to 'high. 

Moderate correlations were found between the Piagetian scores and the 

^ 

attitude test. Low to mo derat^Jjjj^ relations were found between the task 

'scores and scores of the test of science processes. 

Results of the one-way analysis of variance showed differences between the 
means of the three groups, with the formal group scoring higher than the 
transitional group and the ' transitional higher than tne concrete group. 
However, only four of these differences were determined to be significant. 
The F-ratio5 on the Science (STl^P) test (p<.001), the verbal aptitude 
test (p<.01), the matli aptitt^e test (p<.05), and the attitude measure 
(p<. 05) were found to he significant. 



Factor ancflysis idcntJLficd four compQnents, accounting for/6!|v3 percent 
of the 'total variance.^ The^first component, accounting percent 
of the variance, was identified as an achieveinent diniension. The Conser- 
vation of Volume Using Cylinders task*was the only task which loaded on 
this component. According to the authors, this suggests that obtaining 
good grades is probably dependent upon something other than the logical 
reasoning a|)ilities measured by these tasks. The second component was 
identified as a logical reasoning and attitude dimension (13.1 percent). 
The Separation and Exclusion tasks loaded significantly on this component. 
The authors suggest that it js possible th§t as a person develops tl^e 
abilities to successfully respond to the Pia^etian tasks his attitude 
toward science and science teaching as measured by the attitude dnstru- 
ment also improves. A science and mathematics achievement dimension was 
identified as the third component. The highest loadings were on the 
science achievement tests with .85 and .79. Math at^lu^eraent and apti- 
tude loaded at .53 and .45. respectively^ while En^ish achievement and 
verbal aptitude loaded at .37 and .45 respectively. 



Interpr^etations 

The authors conclude that Piagetian measures of formal operational 
reasoning abilities are significantly relateci to achievement, aptitude, 
attitude, and knowledge of the science probesses. They also conclude 
that the Piagetian tasks are generally, but not entirely, content free. 
The authors interpret the implication of their findings by suggesting 
that those secondary school and college students limited to concrete 
modes of thinking should first be confronted with ideas and materials 

at their own level of canability and then /gradually be aaked to deal 

_ < 

with ideas at a more abstract or formal level. 



ABSTRACTOR'S ANALYSIS ' ' . 

This ex posL facto study js^upports previous rescarcji findings indicating 
little or no relationship between science "'knowled(>e and formal «ope ra- 
tional tasks. The results also contribute to existing research by 
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providing further evidence of the construct validity of selected Piagetian 
Styled, formal operational tasks. The strength of these findings have 
been evaluated in terms of the 4tudy's iprocedural aad tfxtemai validity • 



Procedu ral Validity 

Through hypothesis testing, the authors' have avoide.d the unsystematic 
search for relationships so common to most etc post facto research. 
Instead, they have designed a controlled inquiry by predicting the signi- 
ficant and non-significant relationships they would 'have expected if their 
hypothesis were tenable. Procedures such as reporting the reliability of 
the Piagetian tasks, and limiting correction for attenuation to only those 
correlations reaching the .05 signifi^calice level, enhance. tt?e study^s pro- 
cedural validity. The analysis is appropriate and well reporWd'but is 
not as complete as it could be. Post ht)c analyses of the significant F 
ratios found in the analysis of variance are omitted. , The authors' 
reported significant differences among ihe overall mean scores of the 
concrete, transitional, and formal groups on four of the measures. , They 
also report that, in all cases, the- mean scores of the concrete group were 
lower than those of the transitional group and the mean scores of the 
transitional group were lower than those of the formal group. The signi- 
ficant F ratios indicate , that differences exist among these three sets of 
,mean scores. They do not identify vffiere. these differences occur. Multi- 
ple comparison techniques shoulc^ ha^e"been emoloyed to identify where the 
significant differences could be foiind.. The fact that the three sets of 
mean scores fell in th^order the^ didy, provides evidence . that -the - , 
differences ar^ of the nature Kypothesi^^ed. Ilowey^r, multiple comparison 
procedures could have provided more information on "the' strength of this 
evidence. For example, below are three possible results of mul-tiple com- 
parlson analys-is of these data: ' S? ^ « ' 
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A) 

B) 
C) 



Formal 



Formal 



Transitional 



Transitional 



Concrete 



X ^ 



Concrete 



Formal Transit rSnal Concrete 



(means not underlined by the same line differ significantly) 
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In this .example, C provides the strongest evidence that 'the results are 
as called for in the hypotheses. Th/TTe^ults of B provide l^s evidence 
than C but more than A. O ^--^ 



Despite the omission of multiple comparison procedures the results of the 
three different analyses; correlation, one-way analysis of variance an^ 
principal component analysis combine to provide strong arguments and 
logical validity for the study 's\conclusions T 

^ ^ \ . 

External Validity f ^ 

The heterogeneity of tasks, testing procedures, scoring and sample popu- 
lations in formal CJrperatidnal research has made generalizability of 
findings extremely tenuous. Possible sources of threats to the external 
validity of this study are: the selection and scoring of tasks, and the 
sample population. 

Tasks. The conservation of volume using inetal cylinders task has been 
criticized for not testing the concept of conservation of * displacement 
volume from a -Piagetian standpoint (1977). Surprisingly, the results of 
this study provide more support for the use of this particular task than 
for one of the more traditiona'l tasks of displacement using clay. 



The authors indicate they used testing procedures elaborated by Piaget 
and Inhelder (1971) and K^plus and Ljivatelli (1969) . These two groups 
o£ resear chers use different criteria for task responses. Piaget's testr 
ing pjrocedures make use of an objective response as well as rigorous 
questioning of the subject^to determine his justification. The latter 
group generally only require an objective response .or an objective 
response with a brief comment of justification. The differences between 
these procedures can have a profound effect on conclusions (1975). 

Sample. The authors state that they*have no reason to suspect that in 
other samples of coller^e or high school students similar positive corre- 
lations woul/d not be found. However, lack of adequate information about 
the subjects raises questions ns to, their representativeness. The*^'"**^ 
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description of_jJfe sample is limited to tlie subjects* iiiean age and the 
fact that they were freshman and sophomore elementary .education majors. 
No infonh|j^on is given about previous e:meriences of the subjects or 
how they were selected. Is it safe to assunte that this higlily select 
population of education maj^ors had no previois contact with Piagetian 
theoryy? Previous experience with the more cl^mmon Piagetian tasks ^could 
explain their highly skewed response on the, conservation of volume using 
clay task. Information reconstructed from the tables raises other ques- 
tions about the general representativeness of the-subjects. The sample 
size is 71. However, the degrees of^ freedom on the analysis of variance 
table indicates 38 scores on the science achievement test. A 'note after 
the description of the science test states six students elected the 
biology achievement examination, the remaining elected the chemistry, 
examination. How representative is a sample of 38 subjects of whom, when 
given a choice between taking a biology or chemistry test, 32 elect the 
chemistry test? 

While some threats to exteimal validity of formal operational studies can 
be eliminated by detailed research reporting, others resulting from the 
early state of the art in this area are unavoidable^ Until tfiere is 
more Standardization in formal operational res^rch, jgeneralizability of 
results will be extremely limited. 

Through ex post facto research methodology this study idecvtifies the 
existence of relationships between formal operational reasoning and 
achievement, aptitrn^s and attitudes. In addition to contributing to 
the^construct valMity of formal operational development, it identifies 



relevant areas anf variables worthy of further study. The search for 
relationships iri formal operational studies, characteristic of the early 
stages in any 'research effort, should now give way to experimental inves- 
tigations. The structural <^mponents of formal operations, how they 
develop and the variables which affect them are identified directions 
for further experimental researcli. 
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-.Hasan, Omar E., and Victor Y. Billeh. "Relationship. Between 'Teachers' 
' J .Change in Attitudes Toward Science and Some Professional Variables ."y' 

^ ^ Journal of Research in 'Science Teachijig , 12(3) : 247-253 , 1975. 

Descriptors — Attitudes; Educational Research; '^Measurement; 
^Science E4ucation; -Secondary School Teachers; ^Scientific 
Attitudes; Sv^mmer Workshops; -Teacher Background 

Expanded Abstract* and Analysis Prepared Especially for I.S.E. by Frank 
A. Smith, West Chester State College. 



Purpose f 

This investigation was an attempt to determine if a four-week summer train- 
ing course in science teaching \ad an-^^e^^^ect on science teachers^ attitudes 
toward science and also whether certain teacher variables had -an effect on 
these attitudes. The following null hypotheses were investigated: 

Hypothesis 1 ; There is no change in teachers' attitudes toward 
science as a result of a four-week summer training course in , 
science teaching. 

Hypothesis 2; There is no significant association between change 
in teachers' attitudes toward science, as a result of a four-week 
summer training course in science teaching^ and the variables: 
number of years of science teaching experience, number of years - 
of college education, and number of weeks of relevant profes- 
sional ins\rvice training. ^ 



Rationale 

The developmend of favorable attitudes/toward ^cience is generally agreed ^ 
upon to be an important characteristic of a scientifically literate 
person. The authors adopt the assumption that students' attitude^ 
toward science are related to their teachers' attitudes. , Therefore, . 
*it is desirable to*Hnow what variables are related to favourable teacher 
attitudes. * ' ' 
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Research Design and Procedure 

The sample used in the study consisted of 129 secondary science 'teachers 
in Jordan who* had been invited to attend a four-week training course in 
science Reaching. The training course consisted of lectures and demon- 
strations in scientific concepts, teaching methods, and the nature of 
science, laboratory investigations, reading, and related films. The 
dependent variable, attitudes toward science, was measured by an attitude 
scale written by one of the authors. This instrument consisted of 32 
items which were previously rated on a scale of l-Xl by 46 university 
science professors. The split-half reliability of the instrument was 
measured to be .62. The independent variables of the study were the train- 
ing program, the number of years of science i^fcching experience, the number 
of years of college education, and the number of weeks of relevant profes- 
sional inservice training. The research design was of the one-group 
pre test-post test type. Mean scores on the attitude scale were calculated. 
The effect of the training program was determined by comparing pretest and 
posttest scores by means of a t-test. The effect of the various teacher 
variables was determined by multiple regression techniques. 



The inv^tigators found that there was no significant difference between 
pretest and posttest scores on the attitude scale as a result of the train- 
ing program. The* analysis did show, however, that two variables: previous 
professional inservice training and level of education were sigxiif icantly 
(p < .01) related to teachers' gain sjcores on the attitude scale. 



Findings 



On the. basis of these findings the investigators concluded that: 



1. the four-week training pr<^^am was ineffective in producing 
a significant positive change in teachers' attitudes. 

2. teaching cfxperlonce was not significantly related to 
teachers' changes in attitudes. 

3. there was a positive relationship between previous inservice 
training and die teachers' change in attitude. 



/ ■ ■ 



4. . thare was a negative relationship between level of ediica- 

/* ♦ 
' ti5n and teath^rs' change in attitude. 



» 
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Interpretations < 

In a (^iscussion of these conclusions the investigators suggested that the , 
positive relationship between professional inservice training and teachers' 
change in attitudes could bg explaine'd as follows: 

a) the professional inservice training to which these teachers 
had been exposed included lectures on the nature of science 
and the viewing of scientific films. Teachers with more 
exposure to such training wight be expected to show more 
positive attitudes toward science. 

b) the sample was largely volunteer and perhaps teachers who 
volunteer for such programs tend to be more susceptible ' to 
positive changes in attitude. . v 

The negative relationship between level of education and teachers' change 
in attitudes suggested to the investigators that ^those with higher levels 
of education might have formed a stable structure of attitudes towards 
science which would be hard to change, while those of lower le^vels might 
not be stabilized yet^ 



ABSTRACTOR'S ANALYSIS 

Tlie following comments and suggestions about the methodology and findin^'gs 
of >this investigation are made. It is regrettable th"at the research design 
could not have included a control group and that the sample could not have 
been randomly selected. Because of this lack, any generalization of the 
findings of this research beyond the sample used in the study must be 
apptoached with caution. One wishes that there was m^^re information in 
the report about the validity of the instrument used to measure attitudes. 
Data on the results of the administration of the instrument to the group 
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of university ^professors would be helpful. Also, the low reliability 
of the instrument (.62) for the sample raises further doubts about 
generalizing the findings. 

One of the variables is variously referred to as "number of years of 
college education," "level of education," and "university graduate" 
versus "nonuniversity graduate." It is difficult for the reader to 
determine what was actually measured for- this variable because of this 
inconsistency. In Table V on page 252 there is an error in the- record- 
ing of the difference between the pretest score (16.94) and the posttest 
score (16.59) for the university graduates. The difference is reported 
as .1 whereas, if the pretest and posttest scores are corriect, the differ- 
ence should be .35. This increases the value of t from .4 to 1.4. This, 
new vajue of t is still not significant and the findings and conclusions 
remain the same. ' " ^ « 

. 

An interesting question arises when one compares the finding that the 
four-week training course was ineffective in changing attitudes to the 
finding that prior professional inservicS trarining was posiftively' related 
to a change in attitudes, \fliy is prior inservice training related to a 
change in attitudes when this most recent inservice training is not? 
Perhaps the changes are cumulative or related to another variable not 
measured in this research, such as age. These questions might be an area 
of further research. The finding that teaching experience was not related 
to a change in attitudes is also inteil'est ing when compared to the positive 
relatationship found for inservice training: One might expect a relation- 
ship between these two variables since those with the greatest number of 
years of teaching e^erience would seem most likely to also have the 
larger amounts of inservice' training. The question that arises here is 
■how many in "the sample had prior inservice training. 

Perhaps the most interesting finding is the significant negative relation- 
ship between the leyel of education and the teachers' changes in attitudes. 
{ The investigivtors point out that a similar result was obtaiiHd by Hughes 
in a study involving elementary teacher trainees. They suggest that this 
result can be interpreted by assuming that - university graduates have 
stabiliEcd their attitudes toward science in com|5arison to nonuniversity 
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graduates who have not yet stabilized their attitudes. Those with the 
least amounX of education may have the mos^t to gain. This finding can 
have important implications for the selection of participants for similar 
inservice training. 

One can, of course, suggest alternate explanations of this finding. . * * 

Again, age of the subjects seems a possibility. Are the university 
graduates older or younger than the nonuniversity graduates? 

In summary, some of the findings of this research should be investigated 
further. Caution should be used in generalizing the results of this 
study because of some weaknesses in methodology. 



/ 
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Teacher Participation in NSF Institutes and ^tudent Achieveiiient*" 
Journal of Research i n Science Te aching, 13(5) :431~439, 1976, 
Descriptors — "Mchievement ; ilducational Research; Inservice ^ 
Education; '^Inservice Teacher Edlication; Science Education; 
^Science Institutes; Secondary School Science; ''^Teacher 
i Education; Teacher Improvement 

Expanded 'Abst<ract and Analysis Prepared Especially for I.S.E, by Edward 
J. Davis, University of Georgia. 

? 



Purpose ' 

This study was directed at the question, "Is there any evidence that pre- 
college student cognitive achievement has been increased because of 
teacher participation in NSB^sponsored institutes?" It should be noted 
that this study was conducted as a NSF-sponsored project. 



Rationale 

The authors make the following argument for a post hoc analysis: 

An experimental comparison between students whose teachers had attended 
institutes and students whose teachers {lad not would be optimal. The 
experiment would require random assignment of teachers to instit-utes 
(or notX, an^ random assignments of students to teachers. Since NSF has 
not followed such a strategy, * post, hoc comparisons 'may be confounded by 
certain demographic and personological differences between t^^hers who 
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potential factors can never be discounted in a post hoc analysis, but 
those theoretically mostjrelevant should be desalt; with (p. 431). 



^ ResearcH Design and Procedure 

*■ • 

A post hoc analysis was performed. The Authors identified science (or 

t 

mathematics) achievement of tcacliers, and the level of classes to which 
a teacher is assigned to be the theoretical and relevant threats to 
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examining the relationship between teacher institute' j|ttendance and 
students' academic improvement. 

An urban-rural sample of junior and senior high schools was selected for 
science from VJyoming, South Dakota, and Mississippi and for mathematics 
from Calif orMa and Indiana. Urban representation was small. Eighty- 
one percent of the science and- 91 percent of the mathematics classes and - 
teachers came from small towns and cities under 50,000 population. 
Within each school the |xrincipal was asked to select randomly one science 
(or mathematics) teacher and then select randomly one class from this 
teacher's load. This yielded a total of 346 science teachers and their 
classes and 211 mathematics teachers and their classes. Each teacher 
was given an achievement test in the subject area (NT£ exams in either 
^ Physics-Chemistry-Science or Mathematics). Science students topk a 40- 
item test' taken from the NAEP science test and the mathematics stjudents 
were given 40 items from the NL^MA item pool. Different 40-item forms 
were developed for junior high and senior high classes. Not all students 
took *these achievement tests. Each teacher VAas given instructions to 
assign randomly attitude, process, and achievement instruments. 

From a background questionnaire teachers were classified as having NO, 
LOW (1 or 2 institutes attended), or HIGirievels of participation in 
NSF institutes. This placed 36, 36, and .28 percent of the science 
teachers and 43, 29,- and 23 percent of the =mathematics teachers in the % 
respective groups. ; ^ 

The procedures above provided the investigators with a means to coi\trol 
teacher achievement and level of class assignments which were identified 
as obstacles to examining NSF institute participation and student a-chieve- 
ment. Teacher achievement on the NTE exams was used as a covariate in / 
-analysis of .student achievement. The random selection of teachers and 
Classes was used to produce a situation wherein approximately equal ^ 
high-^, middle-, propdirtions of high-, middle-, and low-ability classes 
''appeared in the NO-LOW-HIGH partition of the, teachers. Tlie authors state: 

The possible differential assignment of institute attenders to 
higher-ability classes was examined by testing the independence 



of NSEL^rticipation from the teachers' assassment of tho. ability 
group of the class from which the aahievement dnta were drauni 
(high ability, average ability, low ability, and mixed ability 
groupings). Also tested within the senior high school science 
data was th'e independence of type of class (biology, chemistry, 
and physics) from NSF participation. The chi-square 'statistic 
was used for each test, . . . All chi-square statistics were non- 
significant .at p = .05, indicating inde^5^ndence of the distribu- 
tion of teacher assignments by ability grouping, or subject matter 
in science, from NSF institute participation (p. 433). 



Findings . 

It was reported that: - c"'^ 

The marginal means of student achievement for NSF participation 
show a consistent trend in the direction of better student per- 
formance with increased teacher NSF participation for all four 
analyses • • These means are essentially unaffected by adjust- 
• ment for the .covariate, since none of the regressions are signi-* 
ficant at p = .10.... The honsignif icance of the covariate 
implies that teachers' science ability is not^^elated^ with their 
students ' achievement (p . 435) . 

t 

To follow up differences in mean scores, two planned orthogonal contrasts 
were performed on the senioi; high science scores and two more on senior 
high mathematics scores. These contrasts used an F statistic. The 
first c^onsidered the combined scores of students of ^jOW and HIGH vs .5 
NO^ teacher- institute participation. The second contrast , compared the 
scores of LOW vs. HIGH participation. Three of these four contrasts had 
'significance at the .01 level. These are reported as suggesting that 

teacher attendance at institutes is associated with higher student per- 
• formance than no attendance, and that students whose teachers attended 

the higher number of institutes (more than 2) did better than students 

of teachers attending only 1 or 2 institutes. 



Conclusions 

The authors conclude that a real institute effect is present. They pre- 
scribe that institute attendance bo required of all secondary science and 
mathematics teachers. 
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ABSTRACTOR'S ANALYSIS 

— (T 



This study investigates an important area. In terms of time and money, 
a great deal is being and has been invested in in-service education. 
Student achievement is seldom used as a criterion to evaluate in-service 
programs. It is relevant to do so. 

I am left with some questions, Jiowever. \^en principals are contacted 
is it likely that they will select a science or mathematics teacher (and 
one of their classes) at random? Or will a principal tend to choose a 
teacher and a class according to spme preconceived criteria in %pite pf 
guarantees of non-identification of participants? What about the levels 
of difficulty of the achievement tests? Were they constructed to reflect 
the range of cognitive behaviors identified in the NSLMA study (ComputatiJOn - 
Comprehension-Application-Analysis)? \^at about the attitude and process 
measures? How were they const^ructed?' How did the students perform on 
them? * 

Were these my only concerns, I would feel good about this study. However, 
I must take exception to* the authors' conclusions and recommendations. 
The trend is*for students having teachers who participated in NSF-sponsore( 
institutes to have a significantly higher mean score than students having 
teachers who did not attend institutes. But how much higher^are these 
means? About 1 or 2 points (items) on one 40-itera test. With a large( 
sample it is possible for such a small -mean difference to be significant. 
Statistical significance is present but it is questionable whether this 
difference is meaningful or possessing any practical signif icance, WheiT. 
one considers the cost 'of an institute, to both sponsors J^d participants, 
a recommendation that teachers be required to attend them, based primarily 
on gains of 1 or 2 points on one 40-item instrument seems at best prema- 
» ture and at its worst feathering one's nest. 




RESPONSES TO ANALYSES 




--yl RESPOUS?: TO THE ANALYSIS OF 

Mayer, Victor J., John Disinger and Arthur L. White. "**E>%lua tion of an 
Inservice Program for Earth Science Teachers," by H. A. Smith. 
Investigations in Science Ediication , 2(4) : 12- 14 , 1976 . 

by 

Victor J. Mayer and Arthur L. White 



H.' A. Smith, in r||Viewing the article by >layer, Disinger and IThite in 
Volume 2, No. 4 (1976), makes several points that we feel are open to. 
challenge. He suggests that the study is weak conceptually and the 
results obtained could have been predicted a prio^ri . The study was an 
eKTaluation of a combined shimmer and inservice program to update teachers^ 
knowledge and classroom skills". It is therefore difficult to understand 
what Smith means by coriceptually x^7eak• Perhaps he fails to distinguish 
between a research study* which should have some underpinning of theory 
and evaluation which seeks to demonstrate the relative effectiveness of « 
a program* The program evaluated had a consistent thread or focus. The 
instruments 'use,d were selected or developed because the7 reflected the 
coiften't and philosophy of the ESCP program ^iM/or the prevailing philo- 
sophy underlying modern junior high science curricula. The instruments 
selected therefore were consistent with the objectives of the program and 
the design sli^||^ reflect th^ underlying philosophy of the program. ^ 

»< 

In siiating that the resists could have been predicted a priori he places 
a grea't deal of confidence in the*ab^ity of the urogram developer or he 
misses the ^objective of an evaluation program. The example used in 
support of his statement is that one would logically expect teachers to. 
* gain geological knowledge from a four-week exposure to full-tiine ins true- 
tion. If the progfkm is a goo4 one and the teachers are motivated properly, 
^''this .should indeed by true, 1 Tlie purposp of an evaluation is to see if in 
fact it happened. In point of fact it did not happen with two teachers; 
dne demonstrated a ncgJK:ive gain and one a very^ slight positive gain in 
knowledge' related to the- ohiecl ives of the ESCP. 
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His comments that the analyses do not separate curriculum and teacher 
A effects (we assume he means teacher training, effects) are valid, Tliis 
\ could not be done because 'the nopulntion was t6o small to run the neces- 
sary analysis and the resources available (as is often the case in 
evaluation studies) were not sufficient to set up the neces^silry. compar- 
ison groups. This from our point of view is the mait)r deficiency in the 
study. Were this study to be done again, in the light of the increasing 
* maturity of the field of educational evaluation (this study was started 
almost 10 years ago), a much g're^ter effort would be made to identify a 

larger number of teachers so that a comparison group (or groups) would 

t 

be^ available and to distinguish between those using ESC? and traditional 
approaches during the three-year term of the study. Concurrent with this, 
an attempt would have to be made to obtain the necessary funding. 

VJe concur in the disapp(fintment expressed by &mith in the results of the 
assessment of st;udent achievement during tWrirst year of the study. 
It reinforces the need for such evaluation studies anxi contradicts his ' 
• statement that results could have been predicted a priori . If we were , 
to attempt a priori predictions we Would conclifde that with such better- 
prepared teachers the students would show significant gains ^ in science - 
concents (stressed by the ESCP) as *well as in facts. , This •Was clearly 
not the case.. Following the second year, however, students' did show 
significant gains in both understanding of concepts and of processes of 
science. It may be that prolonged contaot between program 3taff and < 
teachers is necessary before the program has its effect upon curriculum. 

r 

It is disappointing that Smith apparently did not read far enough in the 
article to come, across the renort^f these second year gainsi. It Is 
these gains that the conclusions of the study ar£ based upon. Not the 
first year gains, or lack thereof. ♦ 

Smith contests the use of teachers as rather than students. Since 
teachers, their training and consequent influenc^ on the classroom are 
the focus of the studv (teachers were ^enrolled in the inservice program, 
not students) it is incWed the^teacher »that is the appropriate N. If 
^students had been used, as is common and inappropriately done in many 
studies in science education, all student gains (even the first year 
concept attainment) would 'have been insignificant, merely because of the 
much highcrjN. In this, studv, because of the nature of the program, the 
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appropriateness of the teacher as N is obvious. Me. would submit, however, 
that because of: the uniqueness of a class envj.ronment / all studies that 
involve learning in a class situation should fete the class (and perhaps 
even the teacher in some cases) as M. Even students in different classes 
of the same teacher are exposed to totally different lOviming environments • 
Research and evaluation designs must be^in to recognize these differences y 
and account for them. ^ 

In summary, we are disappointed that the analysis of the st^dy seems so 
superficial. He seems to view it from the arm chair res^^Trcher ' s point 
of view rather than that of the practicing evaLuator, This study is only 
one of two published studies that we know of that has attempted a longi- 
tudinal evaluation of a science inservice program and the only one extend- 
ing beyond the termination of the program* The review seems to have 
entirely missed this point. It is hoped that published studies, even if 
imperfect like the one discussed here, will spur others to conduct similar 
but improved studies and report on their results. Only when the results 
of many such studies are available tJill we be able to document; tfhe effi- 
cacy of different approaches to inservice education. To us that is the 
importance of this study and we are distressed that it was ^ot identified 
as such by the reviewer. *v. 
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